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To better understand ways to improve education, large 
longitudinal data sets are required, such as the National Education 
Longitudinal Study of 1988 which follows students through their 
schooling and into the labor market. In addition, data comparing the 
performance of state and local school systems is essential for 
educational accountability. The following data collection activities 
are needed: (1) data that accurately compare states and local 
education agencies; (2) measures of educational outcomes; (3) 
achievement test scores by subject, age group, minority status, and 
state; (4) performance on minimum competency tests for high school 
graduation and changing requirements, by state; (5) changes in test 
performance of particular cohorts, by state; (6) achievement test 
scores and gains, adjusted for each state's population demographics; 
(7) assessment of higher order cognitive skills; (8) international 
comparisons; (9) assessment of study habits; (10) assessment of locus 
of control; (11) measures of the economic outcomes of schooling; and 
(12) development of causal models of educational improvement. 
Additional data would also improve NCES's Current Population Survey 
on attendance and completion of schooling. (GDC) 
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I. INTRODUCTION AND SUMMARY 

Since education is a local responsibility, the purpose of federal data 
collection is to help the citizenry and political leadership of individual 
states and localities to make better decisions. Better decision making at the 
local level requires two things (1) better understanding of how schools influ- 
ence learning and how effective schools develop and evolve and (2) comparative 
data on the performance of state and local educational systems so the citizenry 
and public officials can hold local school administrators accountable. The 
collection and publication of data is a public service that benefits everyone 
in the country and thus is properly a federal function. 

Achieving the first goal—better understanding of how to improve 
education—requires large longitudinal data sets like the National Education 
Longitudinal Study of 1988 (NELS88) which follow students through their school- 
ing and into the labor market. What is needed is large data sets which merge 
many different kinds of data: 

§ achievement tests for a great variety of subjects (not just one 
subject at a time) which have separate subtests for higher order 
skills and basic skills 

• aptitude tests 

t school records on courses taken, grades, absenteeism, discrip- 
tions of special education service received, and test scores 

• questionnaires measuring student background and attitudes (i.e., 
locus of control) 

• parent questionnaires 

• surveys of the students, teachers, guidance counselors and prin- 
cipals 

• multiple administrations of tests and surveys about 2 to 4 years 
apart 

• follow-up data on labor market outcomes with complete histories 
of employment and unemployment up to age 40 

§ interviews with employers to measure skills and job performance 
of recent school leaners. 

The public needs to be able to monitor the progress and achievements of 
their state and local systems of education. The logical point of comparison is 
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the progress and achievement? of other states so the primary focus of NCES's 
core data program should be on providing data that is comparable across states . 
Emphasis should be placed on measuring outcomes rather than inputs. The key 
data elements that need to be added to the program are the following: 

c Performance on achievement tests in a variety of subjects that 
all (or almost all) students in the school take. (It is feasible 
to put different tests on a common metric and the effort should 
be made.) 

• Comparisons of a states relative performance in tests for first 
graders with performance in later grades. (This is a better 
measure of educational effectiveness than mean test scores which 
are heavi ly effected by the social background of the student 
body. 

• Separate test scores for basic and high level skills. 

• State comparisons of the economic outcomes of schooling: earn- 
ing, employment, occupation. 

• Time series and regional data on salary offers to recipients of 
associates degrees by area of study. 

• Time series and regional data on the economic outcomes of school- 
ing from an improved Current Population Survey. 

• Time series and regional data on how schools influence che devel- 
opment of character (e.g., locus of control). 

§ International comparative data on achievement in particular 
subjects and on time use. 
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II. NEED FOR DATA THAT ACCURATELY COMPARES 
STATES AND LOCAL EDUCATION AGENCIES 



Responsibility for and administration of elementary/secondary education 
lies in state, local and private hands. It is important to remember this fact 
when planning a federally sponsored system of data collection on elementary and 
secondary education. The primary purpose of the common core data program is, 
therefore, not to help the federal government make better decisions but to help 
the citizenry and political leadership of individual states monitor the prog- 
ress and achievements of the educational system in that state. Progress and 
achievements can be defined only when some standard of comparison is available. 
Since the natural point of comparison is the progress and achievements of other 
states, the key requirement for the common core data program should be the 
accuracy of the across state comparisons of the levels (e.g., expenditure per 
pupil in average daily attendance) or changes (e.g., changes in performances on 
standardized tests) in various indicators of effort and performance. 

In order to effectively manage their responsibil ities on a day-to-day 
basis, state governments and local educational agencies (LEAs) need timely and 
accurate data on enrollment, staff, revenues, and expenditures for each of the 
schools and school districts under their charge. From the point of view of the 
state and local administrators, the primary requirement on management informa- 
tion system (MIS) data is comparability across schools and school districts, 
consistency over time and accuracy (conformity to the definitions and concepts 
used in that state's educational legislation and administrative regulations). 
When state legislation changes, the data that needs to be collected will change 
and consistency over time will be set aside in favor of accuracy. 

Historical accident and different needs for data have resulted in MIS data 
that differ in concept and accuracy from state to state. Constructing valid 
estimates of national statistics through the aggregation of MIS data is, thus, 
no easy matter. Since estimates of national aggregates can be obtained where 
necessary by other means (e.g., sample surveys), the common core data program 
should not have the estimation of national aggregates as its central goal. The 
central goal of the common core data program should instead be to provide high 
quality data that are comparable across states. This implies that NCES needs 
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to work cooperatively with the states to insure that data reported are consis- 
tent with the common definitions that have been adopted. If, however, specific 
data elements from some states are not comparable to the data from most other 
states, the data element should either not be published or be published in 
italics with a footnote describing reasons for the lack of comparability. When 
this occurs national aggregates should be reported only for the states that 
have provided comparable information. 
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III. THE NEED TO MEASURE EDUCATIONAL OUTCOMES 

The statistics reported in The Digest of Educational Statistics and The 
Condition of Education are almost entirely measures of inputs into education 
rather than measures of the outcomes of education. Of the 70 tables in the 
1984 edition of The Condition of Education , only 3 displayed data on test score 
outcomes of education and none displayed data on economic outcomes of educa- 
tion. A categorization of the tables in The Digest of Educational Statistics 
(see table I) reveals that it did only slightly better in covering educational 
outcomes. Of the 173 tables in the 1983/84 edition, there were 9 tables dis- 
playing test score outcome data and 9 displaying data on economic outcomes 
(labor force status, occupation and earnings). The tests covered material that 
should be learned before entering college so all 9 of these tables can be 
considered outcome measures for elementary and secondary education. Of the 9 
tables displaying data on economic outcomes, 3 tables addressed the experience 
of bachelor degree holders, 1 the experience of recipients of masters and Ph.D. 
degree holders, 3 the experience of high school graduates and dropouts and 2 
tables presented data on all levels of educational attainment. There were no 
tables devoted to data on the labor market success of completers of 1 or 2 year 
occupational programs or associates degree holders. There were 3 tables on 
high school diplomas awarded and 4 tables on years of schooling completed. 
These, however, are measures of credentials awarded not of learning. The great 
variation in the educational standards across the country and the decline in 
standards over time call into question the usefulness of these statistics for 
comparisons across time and space of the performance of the educational 
system.* 

Table 1 also presents data on changes over time in the subjects covered in 
The Digest of Education Statistics. The number of tables devoted to the eco- 
nomic outcomes of schooling has been stable over time. The number of tables 
providing test performance data rose from 2 in 1969 to 9 in 1983 due to the 
availability of data from the National Assessment of Educational Progress. 

The Effects of Data Availability 

There are important interactions between data availability and the evolu- 
tion of public policy. Statistical agencies respond (often with a considerable 
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TABLE 1 

COVERAGE OF THE DIGEST OF EDUCATIONAL STATISTICS 

1969 1975 1983 

Nat. State Nat. State Nat. State 



Educational Outcomes 



Economic Outcomes 


9 


0 


11 


0 


9 


0 


Test Score Outcomes 


1 


1 


5 


0 


9 


0 


ducational Credentials and Inputs 














Credentials & Yrs of School Completed 


3 


5 


5 


3 


4 


3 


Elementary Secondary Education Inputs 


30 


21 


23 


21 


21 


24 


All Levels Inputs 


11 


2 


12 


3 


6 


2 


nigner LQUCfliion uegrees 


A 
H 


0 


i o 

X v 


0 


A 7 


1 


Higher Education Inputs 


36 


13 


35 


ll 


31 


14 


Adult & Vocational Education 


0 


0 


9 


1 


6 


2 


Federal Programs 


5 


2 


6 


3 


3 


4 


International Education 


6 




6 




8 




Libraries 


7 


1 


8 


2 


5 


2 


R&D 


6 




7 




0 




Equal Educational Opportunity 


5 




2 




0 






123 


47 


139 


47 


121 


52 



170 186 173 



Odd numbered columns are counts of tables which present data for the nation as 
a whole. Even numbered columns are counts of tables which present data com- 
paring states or in a few instances specific institutions of higher education 
or libraries. Input data includes information on students, staff, revenues, 
and expenditures. 
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lag) to *he concerns of policy makers, but in addition, data availability has 
independent effects on the evolution of public policies. 

The availability of statistics recording progress over tine and providing 
geographical comparisons attract public and policy maker attention to a sub- 
ject. The availability of data on inputs—numbers of students, numbers of 
teachers and expenditure—and the lack of data on outcomes contributed to an 
unfortunate tendency during the 60s and 70s for public attention and policy 
analysis to focus on a 1 i mi ted number of measureable inputs into learning 
rather on its outcoiies. 

The Nation at Risk report has made the quality and effectiveness of educa- 
tion the central educational policy question of the 1980s. The kind of data 
included in government publications is changing as a result. But it is doubt- 
ful that the National Commission on Excellence in Education would have had such 
an impact (and might not have even been appointed^, if there had not been data 
available on (1) declinino SAT scores and performance on NAEP exams and (2) the 
poor relative performance of American students in math and science in the 
International Education Agency's studies. The future of the educational reform 
movement will also be influenced by data availability. Since policy is made at 
the state and local level, there is a great need for performance measures that 
are specific to particular states and comparable across states and over time. 

Test Score Outcome Data 

The Department of Education has begun the process of supplying the need 
for comparative data on educational performance by making data on state means 
on the ACT and SAT scores and their changes over time widely available (Wall 
Chart and Indicators). The SAT and ACT tests, however, are curriculum free 
tests of aptitude for college and as such are not appropriate goals for in- 
struction. They are also limited to college bound students. It is important 
that states be able to monitor their progress towards achieving instructional 
goals for all their students and across all grade levels. 

Consequently, high priority should be given to publishing data on the 
following: 

1 . Performance on Achievement tests by subject, by age group (or 
grade), by minority status and by state . Many states mandate 
that all students in the state take particular achievement 
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tests at specified points in their educational career. The 
problem is that different states use different tests. A study 
of the feasibility of linking together the various widely used 
achievement tests has been funded so a methodology for making 
the results comparable across most states will be available 
soon. Disaggregation by subject is important because it helps 
states monitor curriculum reforms that are specific to particu- 
lar subjects. 

2 . Performance on competency tests necessary to receive a high 
school degree and how that has changed over time . 

Consideration should also be given to publishing similar statistics for the 
larger urban school districts. 

Colleges also need to be made accountable; therefore, data on the academic 
performance of their students need to be published. This could be accomplished 
by publishing Graduate Record Exam, SAT and ACT test scores for individual 
colleges and for all of the colleges in a state disaggregated by type of insti- 
tution. Since the mean score will depend on the proportion of a college's 
graduates which take the Graduate Record Exam, it would be essential to publish 
this proportion along with the score. 

Statistics like those in the wall chart titled State Education Statistics 
and those described above will inevitably be used to answer questions like "How 
are we doing relative to other states?" The public and elected officials will 
use such numbers to assess the effectiveness of the educational system in their 
state and to monitor the progress of state reform efforts. The publication of 
the wall chart suggests such uses *re intended. 

There are dangers, however, .ome states are apparently planning to use 
the mean test scores for school districts as a very powerful tool of over- 
sight. 

In the past year, four states—Arkansas, Kentucky, South Carolina, 
and Texas- -have included provisions for academi c bankruptcy in 
their reform bills. The bills in all four states as< that local 
school districts meet certain minimum levels of studert achieve- 
ment and other standards or face the possibility t.iat the state 
wi 1 1 mandate additional reforms for the district, even including 
wresting control of the local school system from the local school 
board—the academic equivalent of putting the district into 
receivership. (Anderson and Pipho, 1984; p. 211) 

If decisions of this magnitude were to be made without taking into account the 
demographic background of the students in the district, great injustices could 
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be done. While state comoarisons of performance could not conceivably lead tc 
the federal government imposing changes on a particular state's educational 
system, they might have powerful political consequences. For example, they 
could influence the outcome of an election for State Superintendent of Public 
Instruction. 

Comparisons of SATs, ACTs or achievement test scores across states prob- 
ably reflect racial and social background differences of their population more 
than they reflect the quality of the schools in that state. Policy makers and 
the public need a statistic for comparing the performance of state educational 
systems that is less subject co such biases. Comparative data that purport to 
measure the performance of an educational system but in fact measure the tal- 
ents and background of the students could confuse the public debate on educa- 
tion more than they enlighten it. Only a tiny minority of the users of such 
statistics will be sophisticated statisticians or educational researchers who 
will estimate multivariate models that control for the student background 
characteristics. Consequently, the data needs to be reported in ways which 
make the comparisons across states as fair and valid as possible . It is there- 
fore essential that someone (NCES, NIE or a researcher under contract) be given 
the task of producing performance indicators that adjust for the social back- 
ground of the students and the talents they bring with them at school entrance. 
The following should be considered: 

3 - Changes in the relative performance of particular cohorts of 
cmidren from particular states . Such a statistic can he 
constructed Dy putting scores from tests administered in the 
first and later grades on a common metric (e.g., standard 
deviation units, grade equivalents or state rankings) and then 
examining how the states performance on this ranking changes as 
grade level increases. While such comparisons might be made 
for different tests given at a point in time, comparisons would 
be more valid if the cohort was held constant. This would be 
done by calculating state rankings on 1st grade scores in 1976 
and then comparing them to the rankings on 8th grade scores in 
1984. Grade equivalents and standard deviation units will 
produce different results. If grade equivalents are to be 
reported, standard deviation units should be reported as well 
and the difference between the two should be explained. 

4. Achievement test scores (levels and gain scores) that have been 
adjusted for the demographic composition of the slates stu- 
dents . Anotner way ot reducing the bias problem rlisru^pH 
above is to estimate statistical models predicting achievement 
as a function of student background characteristics using state 
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aggregate data and then report only the residuals from such a 
model. 

Assessing Higher Order Skills 

Even though most basic skills should be learned prior to the end of junior 
high school, many students arrive in high school without a through grounding in 
basic skills. This fact and great public stress » the importance of basic 
skills seems to have resulted in high school teachers coming to view the teach- 
ing of basic skills as their number one goal. When 10,360 high school teachers 
surveyed as part of the HSB supplementary survey were asked to rank 8 different 
goals of their teaching, 44 percent placed "Basic literacy skills" at the top 
(see table 2). Only 19 percent gave first rank to "Academic excellence or 
mastery of the subject matter of the course." High school should be a place 
for developing higher order skills and reasoning ability such as inference, 
analysis, interpretation, and applying general principals to specific cases. 
The growth of testing and the tendency of most tests to focus on lower rather 
than higher order skills may be contributing to a narrowing of the high 
school's teaching agenda. The National Council of Teachers of Mathematics is 
concerned about this trend and has called for a revision of both the mathe- 
matics curriculum and the tests used to assess progress. It recommended that: 

• "Problem solving be the focus of school mathematics in the 1980s 

•Basic skills in mathematics be defined to encompass more than 
computational ability 

• Stringent standards of both effectiveness and efficiency be 
applied to the teaching of mathematics 

• The success of mathematics programs and student learning be 
evaluated by a wider range of measures than conventional testing" 
(National Council of Teachers of Mathematics, 1980; p. 1) 

Similar problems exist in other subject areas. 

Testing can be designed to assess higher order skills. The recent revi- 
sion of The California Achievement Tests had better assessment of higher order 
skills as an important objective. The College Board and The National Assess- 
ment of Educational Progress have successfully implemented standardized methods 
of assessing writing ability. An effective way to insure that we do not forget 
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TABLE 2 



THE GOALS OF SECONDARY SCHOOL TEACHERS 



Rank 





1 


2 


3 


4 


5 


6 


7 


8 


Basic literacy skills (reading, oath, writing, 
speaking) 




18 


12 


8 


7 


6 


4 


2 


Academic excellence, or mastery of the subject 
matter of the course 


19 


18 


16 


12 


11 


9 


9 


6 


Citizenship (understanding institutions *nd 
public values) 


A 
H 


7 


Q 


it 


J. V 


91 

i- J. 


9? 


9 


Specific occupational skills 


4 


5 


7 


8 


V 


12 


21 


35 


Good work habits and self -discipline 


18 


28 


22 


14 


9 


5 


3 


1 


Personal growth and fulfillment (self-esteem, 
personal efficacy, self -knowledge 


13 


14 


16 


19 


16 


13 


7 


3 


Human relations skills (cultural understanding, 
getting along with others) 


6 


10 


13 


17 


20 


18 


12 


4 


Moral or religious values 


9 


6 


6 


7 


9 


11 


17 


35 



NOTE ; Cell entries are th* percent of teachers who assigned the goal a given 
rank. Except for rounding error each row should sum to 100. Columns some- 
times sum to more or less than 100 because some teachers gave different goals 
the same rank. 



The question was: "If you had to choose from among the eight goals for stu- 
dents listed below, how would you rank them according to their importance in 
your teaching? Enter a "1" for the most important goal, a "2" for the next 
most important goal, and so on, through "8" for the least important goal. 
(FIRST, RANK EACH GOAL. THEN MARK THE MATCHING CIRCLE NEXT TO EACH RANKING.)" 



about the need to teach higher order skills is to publish time series and state 
comparison data from tests and subtests that tap higher order skills. 

International Comparisons of Educational Performance 

New data are becoming available comparing the academic performance of 
students in different countries. NCES should make sure that the resuUs of 
these studies are accessible to educational pol icy makers. 2 A start has been 
made by including data from the Second International Mathematics Study in 
Indicators of Education Status and Trends . The following additional tables 
of International comparison data shou Id be included in one of NCES 1 s 
publications: 

• Mathematics 

- comparisons of performance disaggregated by whether the item 
taps computational skills or problem solving ability; 

- the performance of students from each country that participated 
in the 2nd International Mathematics Study; 

- the changes in math performance between the first and second 
mathematics studies in the U.S. and other countries disaggre- 
gated b' subject and by whether the item taps computational 
skills oi problem solving. 

• Science—The Second International Science Study 

- comparisons separately for biology, chemistry, physics, earth 
science, etc. and separately by whether the item asks for a 
fact or tfp use of higher order skills; 

- comparisons of the changes between the 1st and 2nd Inter- 
national Science Studies disaggregated in the same way 

• Reading Comprehension (Thorndike, 1973) 

• Literature (Purses, 1973) 

• Civic Education (Torney, Oppenheim, and Farnen, 1975) 

• Civic Attitudes (Oppenheim and Torney, 1975) 

• Comparisons of Grade 1 and Grade 5 students in the U.S., Japan 
and Taiwan (Stevenson, 1983). 

There is also a need for the publication of data on the school input and proc- 
esses and environmental variables that may be accounting for the differences 
between countries in achievement. 
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• Time Engaged in Study of Particular Subjects (over the course of 
a student's academic career) 

• Time on Task (percent of time in the school that is actually 
spent engaged in learning) Stevenson (1983) reports observa- 
tional data on classroom time on task and parent child inter- 
action for U.S., Japan and Taiwan. 

• Time Spent Watching TV 

• Number of Books Read 

• Other Measures of Educational Process and Outcomes 

• Measures of the Economic Outcomes of Schooling 

The role of NCES and NIE in international comparative studies should be to 
stimulate others (e.g., the appropriate NIE Centers) to undertake such studies 
and to provide grant support for researchers engaged in such research. Prob- 
ably the most critical need at present is for the collection of internationally 
comparable time use data for both children (in and out of school), parents and 
other adults (Stevenson, 1983; Stafford and Duncan, 1980; and Juster and 
Stafford, forthcoming). 

Assessing Work Habits and Self Discipline 

High school teachers ranked "good work habits and self discipline" as 
their number two goal (see table 2). Sixty-eight percent of the teachers 
reported it ranked among the top three of their teaching goals. Only 53 per- 
cent placed academic excellence in the top three and 43 percent awarded top 
three status to personal growth and fulfillment. Amatai Etzioni has said 

Thus, the root problem is not that millions of high school gradu- 
ates have great difficulties in reading, writing, and 'rithmetric; 
these all-too-common dificiencies are consequences of insufficient 
self-discipline, of inadequate ability to mobil ize self and to 
commit. These graduates enter the adult world twi ce handicapped. 
They suffer both from continued psychic underdevelopment and from 
the inadequate cognitive preparation this underdevelopment helped 
to cause. (Etzioni, 1984; p. 18) 

When 167 employers in the Los Angeles area were asked what was the most impor- 
tant factor in job success, 63 percent picked "good work habits," 23 percent 
picked "technical job skills," and only 14 percent picked "linguistic and 
computing ability" (Wilms, 1983). 

Id 
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Clearly it is important tc keep track of how well our schools are devel- 
oping good work habits and self discipline. High School and Beyond contains a 
number of questions about study habits, class cutting, having work done on 
time, tardiness, attitudes toward study and beliefs about one's ability to 
control one's fate (self efficacy) that can be used to assess character traits. 
NCES publications should present some of this data and where possible make 
comparisons with the Class of 1972 data. It is important that these items be 
retained in NELS88 and that there be no changes in wording. 

Self Efficacy 

The locus of control or personal efficacy scale measuring the belief that 
one's future is controlled by one f s actions is probably the best documented and 
most researched of the sociopsychological scales contained in HSB. It probably 
has a stronger coorelation with what Etzioni is referring to as self discipline 
than any other widely used scale. The scale fits nicely into economic theory, 
for it can be interpreted as a measure of the perceived risk and uncertainty 
surrounding the consequences of one's actions. There is a well documented 
positive coorelation between self eff icacy—believing that one controls one's 
own fate— and labor market success. The analysis of longitudinal data contain- 
ing repealed measures of self efficacy and labor market outcomes has shown that 
an important part of this association is a consequence of self efficacy's 
impact on employment and wages rather than the reverse (Andrisani, 1978; Duncan 
and Morgan, 1979). Kang and Bishop's (1985) analysis of followup data on HSB 
seniors found that self efficacy measured in the senior year hao a larger 
impact on wages and employment in the 2 years after graduation than test 
scores, grades and virtually all other school items except curriculum and 
employment during high school. A one standard deviation rise in self efficacy 
raised earnings of women by 8.1 percent and the earnings o, men by 6.7 
percent. 

Not surprisingly self efficacy also has important effects on learning. 
Using a cross lagged design in which senior test scores were a function of 
sophomore self efficacy, deportment, educational and occupational plans, self 
esteem and performance on verbal, math, science and social science tests and a 
host of other background variables, Hotchkiss (1984) found that a one standard 
deviation improvement in efficacy increased the verbal and civics test score of 
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seniors by 6 percent of a standard deviation and the science test by 5 percent 
of a standard deviation. 

Hotchkiss (1984) has also demonstrated that schools have important impacts 
on the development of self efficacy. In analysis of HSB longitudinal data in 
which self efficacy as a senior was modeled as a function of self efficacy as a 
sophomore, a long list of individual characteristics and a vector of school 
dummies it was found that school dummies exercised significant effects. The 
multiple partial correlation indexing the impact of the school attended was 
two-thirds the size of analogous multiple partial indexing the combined effects 
of personal characteristics and socioeconomic background. 

These results suggest that self efficacy deserves an important role in 
theories of learning and in theories of occupational choice and career attain- 
ment. The results cited above should be viewed only as preliminary forays into 
the development of such theories. Almost nothing is known about how schools 
and teachers foster a sense of self efficacy. NELS88 will be a much better 
'ata set studying these processes than HSB. 3 Consequently it i* critical that 
the self efficacy scale remains in NELS88. 



Measuring the Economic Outcomes of Schooling 

Improved performance on tests of cognitive ability and greater achievement 
in core academic subjects are not the only goals of elementary and secondary 
education. Raising the employability of youth is another important goal. The 
National Commission on Excellence in Education justified its recommendations 
for reform in part by the following: 

Knowledge, learning, information, and skilled intelligence are the 
new raw materials of international commerce and are today spreading 
throughout the world as vigorously as miracle drugs, synthetic 
fertilizers, and blue jeans did earlier. If only to keep and 
improve on the slim competitive edge we still retain in world 
markets, we must dedicate ourselves to the reform of our educa- 
tional system for the benefit of all--old and young alike, affluent 
and poor, majority and minority . Learn i rig is the indispensable 
investment required for success in the "information age" we are 
entering, (p. 7) 

If schooling influences the productivity of the work force as the National 
Commission claims, then people with greater amounts and higher quality educa- 
tion will be more likely to find good jobs that pay higher wage rates. 
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Research on the links between schooling and later success in the labor 
market have fcmd that test scores have rather small impacts on employment and 
wage rates (Bishop, 1985) and that improved performance on standardized tests 
accounts for only a small part of schooling's total impact on labor market 
success (Clneck and Bills, 1982). Vocational education has work attitudes, job 
readiness and occupational skills as explicit goals and the time devoted to 
developing these abilities may reduce the time devoted to the traditional 
academic subjects. Consequently, data on the economic outcomes of elementary 
and secondary education are essential. 

Experience with the Vocational Educational Data System suggests that the 
schools probably cannot be expected to collect and report data on the economic 
success of their graduates that is comparable across schools and across states. 
Data on economic outcomes must therefore be obtained from the Current Popula- 
tion Survey, the Census, longitudinal surveys like NLS, and organizations like 
the College Placement Council. 

Survey of Salary Offers to Associate Degree Recipients 

The College Placement Council collects data from the placement offices of 
colleges and universities on the offers received by B.A., M.A., and Ph.D. 
degree recipients. Northwestern University's Endicott survey publishes similar 
data obtained from a survey of employers. There is no comparable national 
survey of salary offers to the completers of two-year associate degree or one- 
year certificate programs. The Middle Atlantic Career Counseling Association 
(MACCA) has been collecting these data on salary offers received by recipients 
of the Associates Degree from about forty 2 year colleges in their six-state 
area for the last ten years. Their data cover about one-sixth of the nation. 
Data from the VED system is inadequate because the lag in reporting salaries is 
too long and because salary data was not obtained separately for training 
proar*ra of different length. 

After getting support from the American Associations of Community and 
Junior Colleges (AACJC), MACCA and other interested groups NCES should contract 
with the College Placement Council (or an organization like the National Center 
for Research in Vocational Education) to conduct a salary offer survey in as 
many 2 year institutions as are willing to participate. Placement officials at 
participating institutions would report on the job offers received by graduates 
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they provide placement services to. The more rapidly such information is 
disseminated the more valuable it is so semiannual reports should be released. 
Since the labor markets for AA recipients are regional separate reports should 
be published for each of the 10 census regions. The cost of such a data 
collection effort would be under $100,000. 

New Tables Based on Existing CPS Data: Time Series 

The tables in the 1983/84 Digest are useful and should be retained. There 
is a need for new tables, however, that provide comparisons over many years of 
labor market success (employment, occupation, and earnings) for recent school 
leavers. The data proposed would provide a means of tracking whether the 
decline in SAT scores and other indicators of academic achievement have re- 
sulted in a parallel decline in the quality of the jobs obtained by recent high 
school graduates and a rise in their unemployment rates. The following tables 
are recommended. 

1. Unemployment rates and employment to population ratios of young 
people who graduated from or dropped out of high school during 
the previous year and are not enrolled in school (see table C43 
of Labor Force Statistics Derived from the CPS: A Data Bank, 
1982). If possible separate scores for blacks and for single 
females should be published. The table should also contain an 
unemployment rate for prime age (25-55) males and for prime 
age females as a point of comparison. 

2. Indicators of the quality of jobs obtained by recent high 
school graduates and dropouts. The following are recommended: 

• Industrial Composition 

- share of jobs in manufacturing, mining, construction, 
transportation and public utilities 

- share of jobs in government 

- share of jobs in wholesale retail or service 

• Occupational Composition 

- share of jobs in laborers or service occupations 

- share of jobs in operative occupations 

- share of jobs in clerical occupations 

- share of jobs in sales occupations 

These data should be presented both for those who graduated and 
those who dropped out of high school the previous year, and for 
all 18-24 year old high school graduates not enrolled in col- 
lege and for all 18-24 year old high school drop outs. 

3. Average yearly earnings of young people not enrolled in school 
categorized by educational attainment. Three age groups should 
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be reported: 18-24, 25-34 and 35-65. The educational attain- 
ment categories might be 0-11, 12, 13-15, 16 and 17+. Separate 
data should be provided by sex and for full time f ul 1 year 
workers. In order to accentuate the comparisons across educa- 
tional levels, the information could be presented as ratios to 
the earnings of high school graduates (real dollar amounts of 
earnings would be presented only for high school graduates). 
Tabulating these data for recent school leavers and comparing 
it to average weekly earnings of other groups will provide a 
measure of the relative labor market success of those who have 
recently completed their schooling. 

4. Unemployment rates for people categorized by education and by 
age (e.g. , 18-24 and 25-65). 

Data on the labor market behavior of students is also important, though 
not as critical as data on recent school leavers. High school and college 
students spend a great deal of time working. 4 These jobs have both good and 
bad effects on educational outcomes. They help finance college but they do so 
by often reducing the time available for study. 

Improvements in CPS Data 

NCES can greatly improve the usefulness and timeliness of data from the 
Current Population Survey's (CPS) October supplement on attendance and comple- 
tion of schooling by arranging for the addition of the following questions:5 

• usual weekly earnings (same wording as in the January and May 
supplements) 

• highest degree or credential received 

• whether school from which highest degree received was a 4 year 
college, 2 year junior or community college, or technical 
institute 

• whether the school most recently attended (or from which highest 
degree was received) was public, Catholic, other religiously 
oriented private or sectarian private 

• the students major or concentration at that school (for high 
school the categories might be the same as those in question 2 of 
the HSB senior questionnaire, greater detail would be obtained 
from these with some college or a college degree) 

• whether the indi vidual received special education servi ces in 
elementary or secondary school and the type of handicap 
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• stare in which high school was located 

• state in which college was located 

The addition of these 8 questions would greatly improve our ability to track 
changes over time in the economic returns to various types of schooling and to 
compare the effectiveness of schooling in different parts of the country. This 
small addition to only one month of the CPS would go a long way toward meeting 
the Congressionally mandated requirements for data on the outcomes of 
vocational education. 

The monetary returns to col lege attendance and to majoring in specific 
fields vary over time due to shifts in demand and supply for the skills taught 
in these schools (Freeman, 1971 ; 1971a; 1971b) . The monetary I jnef its of 
completing high school and of receiving different types of high school prepara- 
tion also vary over time. Estimates of the impact of vocational education of 
the earnings of 1980 high school graduates (HSB) are considerably more positive 
than estimates in earlier cohorts (such as Class of 1972, the NLS Young Mens 
survey, and the NLS Youth Cohort) (Kang and Bishop, 1985; Meyer, 1982; Daymont 
and Rumberger, 1982). The change could be a result of improvements in the 
quality of vocational offerings but it might also have been a consequence of 
the unique economic environment that greeted 1980's high school graduates. 
Data from repeated CPS surveys are necessary if we are to distinguish long term 
trends in the return to vocational or other particular types of education from 
short run shifts due to the business cycle. The CPS obtains earnings informa- 
tion on a much larger sample (over 70,000 households) than other surveys 
available for studying the impacts of curriculum and private or public control 
of institution (e.g., NELS and NLS) and simultaneous analysis of data from more 
than one year increases the sample size even further. This feature means that 
CPS data can provide reasonably reliable estimates of economic outcomes for 
smaller subgroups (e.g., male graduates of Catholic high schools who did not go 
on to college or female vocational education students in nontraditional occupa- 
tions). The cost of such a supplement would be about $50,000 for data collec- 
tion, another $150,000 for programming, analysis and writing a report. 6 For 
this rather modest investment we would have data on the economic benefits of 
different types of schooling for more than 100,000 working adults. The cost 
would be under 2 dollars per person. 
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It is recognized that the classification of : v'i viduals by their majors or 
field of study would not be as reliable as the data that could be obtained from 
transcripts. Other longitudinal data sets which merge followup data on eco- 
nomic outcomes with high school and college transcript information and baseline 
measures of aptitude, academic achievement, and career plans would still be 
needed. These data sets cost at least 200 dollars per observation, however. 
To be useful there must be data on comparison groups (i.e., those who do not 
take vocational courses) as well as on those who complete the program that is 
the object of intetest. Longitudinal studies like NELS88 cannot be conducted 
on the scale or wit., the frequency that would be needed to construct a reliable 
time series, to make comparisons between the larger states or to provide esti- 
mates of economic outcomes for subsets of the population. Consequently, impor- 
tant as the longitudinal surveys are for analyzing the effects of various 
educational programs the augmented CPS would be a useful supplement. 

Comparing Economic Outcomes in Different States 

Indicators of economic outcomes for specific states are also needed. 7 
Differing economic climates need to be taken into account, so tables must be 
structured so as to highlight statistics that compare the labor market success 
of groups with differing amounts or kinds of education but subject to the same 
economic climate. The greatest need is for tables which describe how particu- 
lar increases in educational achievement (e.g., graduation from high school, 
completing some college, completing a four year degree) improves one's labor 
market success in that state. Since most people find employment in the state 
w/iere they complete their education, these tables will provide a measure (an 
imperfect one to be sure) of the economic payoff to that level of schooling in 
that state—something that should be correlated both with the quality and the 
labor market relevance of that level of schooling in that state. 8 Since the 
source of data for this would be the 1980 Census, the publication of this data 
would not need to be repeated each year. The tables that should be developed 
and made more accessible to state policy makers are the following: 

• Earnings of different age groups (18-19, 20-21, 22-24, 25-29, 
30-34, 35-44, 45-54, 55-64, 65+) by years of schooling completed 
by sex, by minority status and for all workers and full time full 
year workers. (Available in Table 237 of Detailed Population 
Characteristics of the 1980 Census. ) The focuc of the table 
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should be the earning ratios for people with differing educa- 
tional attainment. 

t Occupational and industrial distribution of people categorized by 
educational attainment, age, sex, and minority status. 

• Unemployment rates and employment to population ratios by people 
categorized by educational attainment, age, sex, and minority 
status. 

The usefulness of the 1990 Census for such comparisons would be greatly 
improved if questions were added on the field of study in high school and 
college, degrees received, state in which the individual attended high school, 
and state in which the individual attended college. 
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IV. NEED FOR DATA SETS FOR ESTIMATING CAUSAL 
MODELS OF EDUCATIONAL IMPROVEMENT 



Probably the least controversial conclusion of effective schools research 
is that schools are institutions that have cultures (norms and expectations 
that students and teachers hold in common) which have powerful effects on how 
teachers teach and what students learn. Many of the most important outcomes of 
school ing--verbal ability, reasoning ability, self discipline, character, 
employabil ity, ability to work as part of a team--are the result of the entire 
school experience and not something learned in particular courses. There is a 
need, therefore, for research that takes the school as a unit of analysis. Two 
very important issues need to be addressed. 

• What is it about a school that causes students to learn more, to 
develop better character, or Decome more employable? Associa- 
tions have been found between school climate and learning and 
lists of school effectiveness characteristics have been gener- 
ated. The lists seem reasonable but the research that has gener- 
ated them suffers from serious methodological weaknesses (Purkey 
and Smith, 1982). 

•How are effective schools created? How does a school's ethos 
evolve? Since the school characteristics that are associated 
with school effectiveness are not under the direct control of a 
orincipal or school board, we need to describe the relationship 
between administrative actions and school climate. 

Data for Studies of School Impacts on Learning, 
Character Development and Employabili ty 

The first issue can be addressed if a dataset has the following features 

1. Longitudinal data on students while they are at school. 

- achievement test scores at various points in time 

- aptitude test scores 

- fami ly background 

- educational and occupational plans 

- work habits, study habits, self discipline and self efficacy 

- courses taken from transcripts 

- career goals 

2. Longitudinal data on students after they leave high school. 

- college attendance and completion 

- employment, wages, on-the-job training, productivity, job 
search and turnover 
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- crime and drug/alcohol abuse 

- career goals, locus of control and self esteem 

- job characteristics (e.g., cognitive complexity, autonomy, 
status) 

3. Comprehensive description of the character of the school (in- 
puts, processes, climate) while the student was at the school 

- interviews with principal 

- interviews of teachers and other staff 

- aggregated responses of students describing school climate 

The National Educational Longitudinal Survey of 1988 described in the 
February 1985 NELS88 Position Papers (Longitudinal Studies Branch, NCES 1985) 
has the three features necessary to support studies of how schools effect the 
learning, character development and labor market success of high school stu- 
dents. Below is a 1 ist of comments on speci f ic aspects of the proposed 
design. 

• Substituting data col lection on entering col lege students for 
data collection on graduating high school seniors is recommended, 
for it makes study of college climate and teaching feasible. A 
survey of college teachers should be a part of the NELS88. 

• An attempt should be made to follow up all of the NELS88 students 
after they leave school not just a sample. If cost is a deter- 
rent, less frequent followups (e.g., at 3 year intervals rather 
than 2 year intervals) would be acceptable. Another alternative 
would be to drop the followup of recent college graduates and use 
followups of HSB and NELS88 instead. 

• Many of the most important effects of a quality education do not 
appear until many years later. Followup should consequently be 
continued for 20 years. Once the cohort has reached the age of 
24 the frequency of followup might be shifted to or:e every 5 
years. 

• School records should be accessed to obtain 

- courses taken 

- grades in specific courses 

- test score data on all widely used achievement, aptitude and 
minimum competency tests taken by the student for as far back 
in the students educational career as possible. 

- receipt of special education service and nature of disability 

- absenteeism 

- tardiness 

• The teachers who are surveyed need to be selected in a manner 
which al lows researchers to general ize either to a definable 
subset of secondary school teachers or to the entire population. 
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• Teacher assessments of individual students who are in the HSB 
survey are quite valuable. The checklist the teachers complete 
should include an item on locus of control. 

• Consideration should be given to developing a data set which 
matches students to particular teachers. In a subset of schools 
arrangements could be made to track which teachers each student 
has over the course of high school. In these schools all teach- 
ers would be surveyed and the size of the student sample would be 
increased. 

• The principal and gu ance counselor should be asked whether they 
respond to employer requests for transcript information under two 
different circumstances; the student has signed a release form, 
the student has not signed a release form. Nationwide Insurance 
of Columbus Ohio sent more than 1,200 requests for transcript 
information with student releases to local high schools in 1982 
and received only 93 responses. Not responding to such requests 
when former students have signed a release is a violation of the 
Family Educational Rights and Privacy Act of 1974. 

Sophomore Baseline Questionnaire 

§ The locus of control (self efficacy) scale should be retained. 

t Some students spend little time on homework because they find it 
easy to accomplish quickly. Others spend little time because 
they do not finish the assignments. Therefore it is important to 
ask what proportion of his/her homework assignments are completed 
and handed in on time. 

• Drug and alcohol abuse is a serious barrier to learning and 
questions on these matters should receive high priority. I do 
not agree that these questions are too sensitive to ask in the 
sophomore survey. The questionnaire currently asks about being 
in trouble with the law. 

• The proposed addition of new questions on disciplinary problems 
is a good idea. 

• The question about when was the last time you worked (BB021) is 
not worded well: A question asking what proportion of the school 
year the student has been working is suggested to replace it. 

Employment Data in Followups (applies to NLS72 and HSB as well) 

• Continuous time employment histories as in HSB followups are very 
valuable and should be retained. The number of different jobs 
should not be limited to 4, however. 

• The question on how you found this job is very valuable and it 
should be retained. There is evidence that for vocational stu- 
dents 1 i nformal recommendations by teachers are more effective 
than school place;nent services in finding jobs for students and 
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graduates (McKinney et al. 1980). A distinction between the two 
should be drawn by adding "Through a former teacher" to the list 
of alternatives. Finding a job through a friend or relative who 
works at the firm has an especially strong tendency to reduce 
turnover. The distinction can be drawr. by using the following 
response categories "through a relative who worked at the firm, 
through a relative who did not work at the firm, through a friend 
who worked at the firm, and through a friend who did not work at 
the firm." 

• The question on why you left the job should be retained, but 
involuntary terminations need to be disaggregated into: 

- fired 

- temporary or seasonal job 

- lay off not anticipated when hired. 

• The question on training (51A-D and 52/-E of Second c ollowup of 
Seniors) should be asked for as rr iy jobs as possible and 
especially for the first few jobs aiter leaving school. If a 
tradeoff is necessary, reduce the number or job satisfaction 
questions. 

• The unemployment section is good and should be retained. It 
should include a question about receipt, of unemployment insur- 
ance. A separate section should be developed to ask about the 
search process that led io the first job after leaving school the 
last time. 

• Questions on tardiness, absenteeism and "goofing off" on the job 
that parallel similar questions about school behavior would be 
desirable. 

• Wage rates do not fully capture the productivity impacts of a 
good education. Studies based on absolute measures of produc- 
tivity of incumbents in particular jobs that pay the sane wage to 
everyone with equal tenure find it has a coefficient of variation 
of 20 percent (Schmidt and Hunter, 1983). The respondent should 
be asked to make a self-assessment of their effort on the job and 
of their productivity . Question wording might be something like: 
h kelative to the other employees at the firm, how much harder or 
less hard did you work? How do you think your supervisor as- 
sessed your productivity relative to the other employees doing 
similar work?" If a tradeoff is necessary reduce the number 
questions of satisfaction and probabilies of taking particular 
kind* of jobs. 

• Consideration should be given to o aining permission from stu- 
dents to speak to their employer or supervisor and then asking 
the supervisor for an assessment of how the individual's produc- 
tivity and capabilities compare with other employees. 
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Data for Studies of School Improvement 

Studies of how schools evolve require data sets with repeated measures of 
school climate and school processes. Such data s^ts are very expensive to 
create, however, because thp key climate and process variables cannot be mea- 
sured without interviewing large samples of te<achers and students at each 
school. A recent survey of 10,360 teachers which measured school climate and 
processes at 500 high schools cost more than $200,000 to conduct. Longitudinal 
data on school climate and process can be made available at no additional cost 
simply by returning to these 500 schools in the N ELS88 and repeating some of 
the same questions in the survey of teachers. 

Data for Studies of Learning in Elementary 
schools and Junior High Schools 

Longitudinal data sets that start when the child is very young and follow 
him/her all the way through school and into the labor market would make pos- 
sible a number of very valuable studies: 

• Long term effects of early childhood education and types of 
special education services 

§ Elementary school effectiveness 

• Junior high school effectiveness 

• Retention in high school through the 10th grade. Most dropouts 
leave school before the end of their sophomore year and so are 
not part of the NELS88 baseline. 

• Evolution of career awareness and aspirations prior to 10th 
grade. 

Consideration should therefore be given to, starting in 1988, a longi- 
tudinal survey of second graders and their parents in a sample of the elemen- 
tary schools that feed into the high schools selected for NELS88. The parent 
child data would then be merged with school records on grades and aptitude and 
achievement test scores and su* veys of the principals and teachers of the 
school attended. These students would then be resurveyed in 1992 and 1996 and 
serve as the sophomore cohort of NELS96. 

Another way a longitudinal data set might be developed would be to under- 
take a followup of the 1st through 3rd grade children who participated in the 
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Sustaining Effects Study in 1975 through 1978. An additional requirement would 
be surveys of the principals and teachers in the elementary and secondary 
schools attended by these students and the development of a mechanism for 
matching each student to the teachers that he/she had. Even if no further 
augmentation of the Sustaining Effects data is planned there would be sig- 
nificant benefits to making a well documented version of the data set more 
widely available through NCES. 
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FOOTNOTES 



1. Awarding a high school diploma to a student who cannot read or write, may 
make that individual better off but it depreciates the value of the diploma for 
everyone else and lowers the efficiency of the process which matches individ- 
uals to jobs. 

2. The IEA dats comparing the U.S. and other countries first became available 
in the early 70s. To my knowledge these data neve appeared in an NCES pub- 
lication until after the report of the National Commission on Excellence in 
Education. If these statistics had been appearing regularly in publications 
like The Condition of Education , the nation's attention might have been di- 
re cted~To^fiTl)roI^ standards much sooner than it was. 

3. NELS88 is a better vehicle for studing the effects of school process and 
climate on student outcomes because the school process and climate variables 
will be measured when the students are sophomores rather than 2 years after 
graduation as in the case of HSB. 

4. High school juniors and seniors average more than 10 hours a week in paid 
employment during the school year. Time engaged in a learning activity aver- 
aged under 20 hours (Bishop, 1985; p. 9). 

5. An alternative might be to add the questions about school background to a 
month that already has the weekly earnings data such as May or January. The 
March Survey is already too crowded to be considered for this purpose. 

6. The source of these estimates is an off the record conversation with a 
former Census Bureau staff member). 

7. At present the Digest of Education Statistics does not contain such data 
(see table 1). 

8. The CPS question on state in which one received one's high school educa- 
tion described in the previous section would allow the construction of similar 
tables using CPS data that would assign migrants to the state where they re- 
ceived their education rather than their state of residence. 



30 

86 



REFERENCES 



Andrisani, Paul J. "Internal-External Attitudes, Personal Initiative, and the 
Labor Market Experience of Black and White Men." The Journal of Human 
Resources 12 (3) 1978:308-325. 

Bishop, John. Preparing Youth for Employment. Columbus: The National Center 
for Research in Vocational Education, The Ohio State University, 1985. 

Comber, L. C, and Keeves, J. P. Science Education in Nineteen Countries: An 
Empirical Study. Stockholm: Almqvist and Wikse 11, and New York: John 
Wiley, 1973. 

Daymont, T. N . , and Rumberger, R. W. "The Impact of High School Curriculum on 
the Earnings and Employabil ity of Youth." In Job Training for Youth, 
edited by R. Taylor, H. Rosen, and F. Pratzner. Columbus: The National 
Center for Research in Vocational Education, The Ohio State University, 
1982. 

Duncan, 6. J., and Morgan, J.N. "Sense of Efficacy and Subsequent Change in 
Earnings--A Replication." Journal of Human Resources 14 (4) 1979:649- 
557. 

Etzioni, Amitai. "Self-Discipline, Schools, and The Bit ,ness Community." 
Final report to the U.S. Chamber of Commerce Foundation, 1984. 

Freeman, Richard B. "A Cobweb Model of the Supply and Starting Salary of New 
Engineers." Industrial and Labor Relations Review 29 (January 1976a) :236- 
248. 

_ . The Market for College-Trained Manpower: A Study in the 

Economics of Career Choice. Cambridge, MA: Harvard University Press, 
1971. 

. The Overeducated American. New York: Academic Press, 

vmF. 

Juster, F. Thomas, and Stafford, Frank. "Time Use Survey." In Time Goods and 
Well-Being, edited by F. Thomas and Frank Stafford. Ann Arbor: Institute 
for Social Research, University of Michigan. Forthcoming. 

McKinney, Floyd, et al. Factors Relating to the Job Placement of Former Sec- 
ondary Vocational Educational Students. Columbus: The National Center 
for Research in Vocational Education, The Ohio State University, 1982. 

Meyer, R. "Job Training in the Schools." In Job Training for Youth, edited 
by R. Taylor, H. Rosen, and F. Pratzner. Columbus: The National Center 
for Research in Vocational Education, The Ohio State University, 1982. 



ERLC 



31 

87 




National Council of Teachers of Mathematics. An Agenda for Action: Recom- 
mendations for School Mathematics of the 1980s. 1980 . 

Opoenheim, A. N., and Torney, Judith. The measurement of Children's Civic 

Attitudes in Different Nations. Stockholm: Almqvist and Wiksell and New 
York: John Wiley, 1974. 

Olneck, M. R., and Bills, D. B. "What Makes Sammy Run?: An Empirical Assess- 
ment of the Bowles-Gintis Correspondence Theory." American Journal of 
Education 89 (1) November 1980:27-61. 

Purkey, Stewart C, and Smith, Marshall S. "Effective Schools--A Review." 

Madison: Wisconsin Center for Education Research School of Education, 
University of Wisconsin at Madison, September 1982. 

Purves, A. C. Literature Education in Ten Countries: an Empirical Study. 
Stockholm: Almqvist and Wiksell and New York: John wiley, 1973. 

Stafford, Frank P., and Greg Duncan. "The Use of Time and Technology by 

Households in the United States. In Vol. 3 of Research in Labor Eco- 
nomics, edited by R. G. Ehrenberg. Greenwich, CT: JAI Press, 1980. 

Thorndike, R. L. Reading Comprehension Education in Fifteen Countries: An 

Empirical Study. Stockholm: Almqvist and Wiksell and New York: John 
Wiley, 1973. 

Torney, J. V.; Oppenheim, A. N.; and Farnen, R. F. Civic Education in Ten 

Countries: an Empirical Study. Stockholm: Almqvist and Wiksell Inter- 
national and New York; John Wiley, 1975. 

Wilms, Wellford W. Technology, Job Skills, and Education: A Reassessment of 
the Links. Los Angeles: University of California, 1983. 



ERLC 



32 

88 



